In this article, we will explore various aspects of BERT, including the landscape at the time of its creation, a detailed breakdown of the model architecture, and writing a task-agnostic fine-tuning pipeline, which we demonstrated using sentiment analysis. Despite being one of the earliest LLMs, BERT has remained relevant even today, and continues to find applications in both research and industry.
Researchers from NYU Tandon School of Engineering investigated whether modern natural language processing systems could solve the daily Connections puzzles from The New York Times. The results showed that while all the AI systems could solve some of the puzzles, they struggled overall.
This article provides a beginner-friendly introduction to Large Language Models (LLMs) and explains the key concepts in a clear and organized way.
ColBERT is a new way of scoring passage relevance using a BERT language model that substantially solves the problems with dense passage retrieval.
With deep learning, the ROI for having clean and high quality data is immense, and this is realized in every phase of training. For context, the era right before BERT in the text classification world was one where you wanted an abundance of data, even at the expense of quality. It was more important to have representation via examples than for the examples to be perfect. This is because many Al systems did not use pre-trained embeddings (or they weren't any good, anyway) that could be leveraged by a model to apply practical generalizability. In 2018, BERT was a breakthrough for down-stream text tasks,
